REMARKS 

Claims 1-27 are presented for examination, of which Claims 1, 20, and 24 
are in independent form. Claims 12 and 23-25 have been amended, with no change in 
scope, as to various matters of form. 

An early and favorable examination on the merits is respectfully requested. 

Applicant's undersigned attorney may be reached in our New York office by 
telephone at (212) 218-2100. All correspondence should continue to be directed to our 
below listed address. 

Respectfully submitted, 




'Attorney for Applicant 
Registration No. ^~Z5^ l 2ffi^> 



FITZPATRICK, CELLA, HARPER & SCINTO 
30 Rockefeller Plaza 
New York, New York 10112-3801 
Facsimile: (212)218-2200 
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IN THE CLAIMS: 

The following is a complete listing of the claims, and replaces the original 
claims of the present application filed herewith. 

Claim 1 (previously presented): An image processing device comprising: 

an image pick-up device having the fixed positional relation with a 
measurement object; 

an attitude sensor for measuring the attitude at an image pick-up visual 
point of said image pick-up device; 

a storage unit adapted to store the calculation information to calculate the 
attitude and/or position of said measurement object on the basis of an output from said 
attitude sensor; 

a target image setting unit adapted to set a target image that is an object for 
detecting a predetermined index on the basis of a picked-up image picked up by said image 
pick-up device; 

a detecting unit adapted to detect the position of said index in said target 
image by performing a template matching process between a template image of said index 
and said target image; 

an updating unit adapted to update said calculation information stored in 
said storage unit on the basis of a detected position of said index detected by said detecting 
unit; and 



a calculation unit adapted to calculate the attitude and/or position of said 
measurement object on the basis of said measured value and said calculation information 
updated by said updating unit. 

Claim 2 (previously presented): The image processing device according to 
claim 1, wherein said target image setting unit obtains a prediction position of the index in 
said picked-up image employing said measured value and said calculation information 
stored in said storage unit, creates an image with a peripheral area around said prediction 
position in said picked-up image subjected to a rotational process on the basis of a 
rotational angle in a roll direction of said image pick-up device derived from said measured 
value, and outputs said image as a target image. 

Claim 3 (previously presented): The image processing device according to 
claim 2, wherein said calculation information is the correction information to correct for an 
error in the measured value of attitude measured by said attitude sensor, and said 
calculation unit calculates the attitude of said measurement object on the basis of said 
measured value and said correction information. 

Claim 4 (previously presented): The image processing device according to 
claim 1, wherein said calculation information is the correction information to correct for an 
error in the measured value of attitude measured by said attitude sensor, and said 
calculation unit calculates the attitude of said measurement object on the basis of said 
measured value and said correction information. 
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Claim 5 (previously presented): The image processing device according to 
claim 1, wherein said calculation information is the correction information to correct for an 
error in the measured value of attitude measured by said attitude sensor and the position 
information of the image pick-up visual point of said image pickup device, and said 
calculation unit calculates the position and attitude of said measurement object on the basis 
of said measured value, said correction information and said position information. 

Claim 6 (previously presented): The image processing device according to 
claim 1, wherein said calculation information is the position information of the image 
pick-up visual point of said image pickup device, and said calculation unit calculates the 
position and attitude of said measurement object on the basis of said measured value and 
said position information. 

Claim 7 (previously presented): The image processing device according to 
claim 5, wherein said updating unit updates the position information in the two directions 
except for a depth direction in the camera coordinate system of said image pickup device, 
even when an index of only a single point is detected in said detecting unit. 

Claim 8 (previously presented): The image processing device according to 
claim 3, wherein said correction information is the information to correct for an error in the 
azimuth direction among the measured values of the attitude measured by said attitude 
sensor. 



Claim 9 (previously presented): The image processing device according to 
claim 1, wherein said updating unit updates said calculation information on the basis of the 
detected position of said index in said picked-up image. 

Claim 10 (previously presented): The image processing device according to 
claim 3, wherein said updating unit updates said calculation information on the basis of a 
typical value of the updated value of said calculation information obtained for each index 
when a plurality of indices are detected in said detecting unit. 

Claim 1 1 (previously presented): The image processing device according to 
claim 3, wherein said updating unit updates said calculation information on the basis of a 
dislocation between the prediction position and said detected position of said index in said 
target image. 

Claim 12 (currently amended): The image processing device according to 
claim 11, wherein said updating unit updates said calculation information on the basis of a 
typical value of said dislocation obta^ied for each index when a plurality of indices are 
detected in said detecting [[means]] unit . 

Claim 13 (previously presented): The image processing device according to 
claim 5, wherein said updating unit updates the position information in three directions in 
the camera coordinate system of said image pick-up device, when two or more indices are 
detected in said detecting unit. 
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Claim 14 (previously presented): The image processing device according to 
claim 1, wherein said index is a projected image of a landmark in the real space onto said 
picked-up image. 



Claim 15 (previously presented): The image processing device according to 
claim 14, wherein the position of said landmark in the real space is known. 

Claim 16 (previously presented): The image processing device according to 
claim 4, wherein said index is an image feature on said picked-up image. 

Claim 17 (previously presented): The image processing device according to 
claim 1, wherein said measurement object is an image pick-up visual point of said image 
pick-up device. 

Claim 18 (previously presented): The image processing device according to 
claim 17, further comprising display unit for displaying said picked-up image with the 
image in the virtual space superposed thereon on the basis of the attitude, or position and 
attitude of said image pick-up device calculated by said calculation unit. 

Claim 19 (previously presented): The image processing device according to 
claim 1, wherein said measurement object is a visual point of the observer, and said image 
processing device further comprises display unit for displaying the image in the virtual 
space drawn on the basis of the attitude or position and attitude of the observer calculated 



by said calculation unit on said display screen, while optically transmitting the image in the 
real space through said display screen observed by the observer. 

Claim 20 (previously presented): An image processing device in which the 
position of an index in a picked-up image picked up by an image pick-up device is detected 
by template matching employing a template image of said index, comprising: 

an attitude sensor for measuring the attitude at an image pick-up visual 
point of said image pick-up device; 

a target image creating unit adapted to create a target image having a 
peripheral area around a prediction position in said picked-up image subjected to a 
rotational process on the basis of the rotational angle in a roll direction from said measured 
values by obtaining said prediction position of the index in said picked-up image, 
employing the measured value of said attitude measured by said attitude sensor, and 
outputting the target image; and 

a detecting unit adapted to detect the position of said index in said 
picked-up image by performing a template matching process between said template image 
and said target image. 

Claim 21 (previously presented): The image processing device according to 
claim 1, further comprising a template image creating unit adapted to create a template 
image for detecting said index from the image, wherein said template image creating unit 
specifies the position of the index in the picked-up image when said image pickup device is 
located at a predetermined position and attitude and creates a template image of the index 
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with a peripheral area around the position of the index in said picked-up image subjected to 
a rotational process on the basis of the rotational angle in a roll direction from said 
predetermined positional attitude. 

Claim 22 (previously presented): The image processing device according to 
claim 1 , wherein said attitude sensor measures the attitude of an image pick-up visual point 
of said image pick-up device in a state where there is an accumulated error in the measured 
value in the azimuth direction. 

Claim 23 (currently amended): Theimage processing device according to 
claim 22, wherein said attitude sensor is constituted of comprises a gyro sensor. 

Claim 24 (currently amended): An image processing method comprising: 

an image pick-up step A of picking up an image with an image pick-up device 
having the fixed positional relation with a measurement object; 

an attitude measuring step A of measuring the attitude at an image pick-up 
visual point of said the image pick-up device; 

a storage step a of storing the calculation information to calculate the attitude 
and/or position of said the measurement object on the basis of the measured value 
measured [[at]] in said attitude measuring step; 

a target image setting step A of setting a target image that is an object for 
detecting a predetermined index on the basis of said the picked-up image; 



a detecting step A of detecting the position of said the index in said the target 
image by performing a template matching process between a template image of said the 
index and said the target image; 

an updating step A of updating said the calculation information stored [[at]] in 
said storage step A on the basis of a detected position of said the index detected [[at]] in said 
detecting step; and 

a calculating step a of calculating the attitude and/or position of said doe 
measurement object on the basis of said the measured value and said the calculation 
information updated [[at]] in said updating step. 

Claim 25 (currently amended): The image processing method according to 
claim 23, wherein said target image setting step comprises creating a target image with a 
peripheral area around a prediction position in said picked-up image subjected to a 
rotational process on the basis of the rotational angle in a roll direction of said the image 
pick-up device derived from said the measured value by obtaining said the prediction 
position of the index in said the picked-up image, employing the measured value and the 
calculation information stored [[at]] in said storage step. 

Claim 26 (previously presented): A program code for executing the image 
processing method according to claim 24. 

Claim 27 (previously presented): A storage medium storing the program 
code according to claim 26. 



